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Abstract 


This research sought to analyze the reliability of an instrument to measure teacher knowledge of a 
group of pre-service chemistry teachers participating in the Institutional Program for Scholarships for 
Beginner Teachers in Brazil (PIBID).. Thus, it was analyzed the evaluation of a group of students about 
the Pedagogical Content Knowledge (PCK) of five pre-service chemistry teachers from the program. 
Accordingly, an instrument containing 28 statements and an open question was applied to a group of 70 
students from a public school to whom the pre-service teachers had applied their didactic proposals. The 
purpose of the instrument was to measure the perception of the students’ about the components of the 
teacher knowledge (Subject Matter Knowledge - SMK, Instructional Representation and Strategies - IRS, 
Instructional Objective and Context - IOC and Knowledge of Students’ Understanding - KSU) of the pre- 
service teachers. The results of the analysis revealed that the instrument about teacher s knowledge has a 
high reliability based on the alpha Cronbach values. The results of the alpha coefficient for the pre-service 
teachers were all above 0.9 suggesting that the instrument is reliable and has consistent measurements. 
Regarding to the exploratory analysis, the main scale in the PCK evaluation of the undergraduates was 
four which means that the students consider that the aspects related to the SMK, IRS, IOC and KSU are 
frequently present in the classes of the future teachers. 

Keywords: instrument reliability, PIBID, redox reactions, teaching knowledge, PCK. 


Introduction 


In the scientific literature there are several authors dealing with the knowledge necessary 
to be a teacher. Shulman conceived of a knowledge base inherent in the teaching profession 
(Shulman, 2004). The author proposed seven knowledge categories for teachers (Shulman, 
1987), particularly Pedagogical Content Knowledge (PCK), which represents knowledge that 
distinguishes a chemistry teacher from a chemistry specialist, being specific for each subject 
and developed in practice (Fernandez, 2014). Shulman’s proposals inspired other researchers to 
investigate PCK (Grossman, 1990; Magnusson, Krajcik & Borko, 1999; Rollnick & Mavhunga, 
2014; Van Driel & Verloop, 1998; Fernandez, 2015). 

Socialized studies on PCK mean that the participants of the studies are generally teachers 
who teach in basic education and they are based on a qualitative approach, which suggests that 
research analyzing the PCK of university professors and pre-service teachers is rare. However, 
despite being in early stages, some researchers started to develop questionnaires to measure the 
PCK of teachers (Kleickmann et al. 2013; Kirschner, Borowski, Fischer, Gess-Newsome & Von 
Aufschnaiter, 2016). Among the quantitative studies, though, few analyze student perceptions 
of the teachers’ knowledge of their respective professors (Criua & Marian, 2014; Halim & 
Abdullah, 2014; Jang, 2011; Sofianidis & Kallery, 2016; Tuan, Chang, Wang & Treagust, 
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2000). Jang, Guan and Hsieh (2009) developed a Likert scale type instrument to measure how 
undergraduate students perceived the PCK of their professors. In assessing the results, the 
authors argue that professors began to consider student understanding over teaching strategies 
used in the teaching environment. This instrument was used in other studies (Girotto Junior, 
2015; Jang, 2011; Sofianidis & Kallery, 2016). In one of these researches, Girotto Junior (2015) 
applied it to undergraduate students in chemistry. The results of the instrument were compared 
to the view of the researcher based on one phase of their studies with qualitative characteristics. 
The analyses imply that the students are aware of their professors’ knowledge of practice, but 
they have difficulty in recognizing other types of knowledge, which are part of out-of-class 
activities, such as, for example, class planning and issues related to the prior knowledge of 
the students. Possible inconsistencies between perceptions by students and outside observers 
of teacher knowledge can be a reflection of the different aspects each one observes (Peterson, 
Wahlquist & Bone, 2000). 

Therefore, the aim of the research is to analyze the reliability of an instrument on teacher 
knowledge of five pre-service chemistry teachers who participate in the “Programa Institucional 
de Bolsa de Iniciagao a Docéncia (PIBID)”, Institutional Scholarship Program for Beginning 
Teachers. 


Program for Beginning Teachers - (PIBID) 


The Programa Institucional de Bolsa de Iniciagado a Docéncia (PIBID, “Institutional 
Scholarship Program for Beginning Teachers’) is a relatively recent Brazilian policy that 
promotes and values the teaching career. It enables undergraduate students to have teaching 
experience during the undergraduate course (Braibante & Wollmann, 2012; Gatti, Barreto & 
André, 2011; Scheibe, 2010). Its purpose is to promote the integration between basic education 
and higher education, promoting reflections on the educational practice and to promote 
professional development of teachers (Canan, 2012; Ribeiro & Nogueira, 2016). In this sense, 
PIBID has been considered a third educational space, in addition to undergraduate studies and the 
supervised internship (Felicio, 2014). Research on this program are dedicated to investigating 
mainly the educational aspects of the undergraduates made possible by PIBID. For Silva 
(2015), the program promotes the interest of the student teachers in the course. Other studies 
have analyzed the influence of PIBID in structuring the teaching knowledge of its participants 
(Morais & Ferreira, 2014; Sa & Garritz, 2014) and adopt a qualitative methodology. In the 
exploratory analysis of studies related to PIBID, the studies are associated to teacher training, 
the production of teaching materials and reports of experiences (Passoni et al., 2012), suggesting 
that studies addressing student perceptions on the teaching practice of PIBID students are rare, 
mainly of their teaching knowledge (Freire & Fernandez, 2014). 


Research Methodology 
Context of the Research 


The PIBID students participating in this research were part of a subproject organized by 
a coordinator in the Chemistry teacher education program, two in-service Chemistry teachers 
(supervisors) and eleven pre-service teachers. In the subproject, the teaching proposals were 
associated to the Politica Nacional de Residuos Solidos (PNRS, “National Policy of Solid 
Waste’’) (Brasil, 2010) and founded in the Science, Technology and Society (STS) approach 
and on experimental activities. During the implementation of the subproject, meetings were 
organized and conducted by the coordinator who, together with the pre-service teachers, defined 
the classroom objectives, the theoretical references to be studied and the inquiry questions to 
be chosen for each group of preservice teachers, which guided the planning of the teaching 
sequences. Then, in individual meetings with each group, the concepts, teaching strategies, 
methods of evaluation, among other aspects involved in the teaching sequences were defined. 
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In this research, the teacher knowledge of five pre-service chemistry teachers were 
analyzed. For ethical reasons they will be referred to using fictitious names, Antonio, Melissa, 
Clara, Elder and Mateus. Over the course of a semester, Ant6nio and Melissa developed the 
topic “The inadequate disposal of automotive batteries” using the chemical concept of redox 
reactions, in a senior year of Educagdo de Jovens e Adultos (“Youths and Adults Education’). 
Elder, Clara and Mateus taught classes to sophomore year from secondary school, about the 
issue “Disposal of fluorescent bulbs in teaching electrochemistry”. 


Quantitative Research 


This research used a quantitative approach. The reliability of the instrument was 
determined by analyzing the internal consistency through Cronbach's alpha. According to 
Diehl, quantitative research minimizes possible “[...] distortions of analysis and interpretation 
and allowing greater safety margin” (p.13, 2004). 

The instrumentused in this research was adapted from the one developed by Jang, Guan and 
Hsieh (2009) and translated by Girotto Junior (2015). The instrument consists of four categories 
of statements: Subject Matter Knowledge (SMK), Instructional Representation and Strategies 
URS), Instructional Objective and Context (IOC) and Knowledge of Students’ Understanding 
(KSU). SMK is related to student perceptions regarding their teachers knowledge of the topic 
in the context of the discipline. IRS aims to examine how the students understand the elements 
associated to the teaching strategies of their teachers including group discussions, experimental 
investigation, among others. The purpose of IOC is to understand how the students perceive the 
teacher’s proposals and objectives in teaching, which encompasses knowledge of the context, 
the curriculum and class management. KSU is related to student perception regarding the 
methods of evaluation used by the teacher to evaluate them (Jang, Guan & Hsieh, 2009). Each 
category had six options (Likert scale). The scale ranged as follows: 0 (I don’t remember / I 
don’t know), | (If it never happened in the teaching practice), 2 (If it seldom happened in the 
teaching practice), 3 (If it sometimes happened in the teaching practice), 4 (Ifit often happened 
in the teaching practice) and 5 (If it always happened in the teaching practice). The instrument 
also included an open question that allowed the respondent to talk about a learning difficulty or 
to make some comments on the pre-service teacher. The original test did not have the 0 (I don’t 
remember / I don’t know) option on the scale, which we included in this research. The adapted 
instrument used in this research consists of 28 statements and six options (appendix). 


Procedure 


The instrument was applied in a class where each pre-service teacher taught their lessons. 
The students who answered the instrument were those who attended classes during the school 
year, meaning we worked with a population of convenience, despite the understanding that 
the greater the number of respondents, the greater its variance (Bland & Altman, 1997) and, 
consequently, that the number of participants influences the value of the Cronbach’s Alpha 
calculated. As such, the instrument was answered by 70 students, with 45 from the sophomore 
year of regular Secondary School and 25 from the senior year of EJA (EJA, Youths and Adults 
Education). The instrument was answered by a total of seventy students from Secondary School 
and from EVJA. 


Quantitative Data Analysis - Cronbach’ Alpha 
In this research, Cronbach’s Alpha was used to analyze the reliability of the instrument 
(Cronbach & Shavelson, 2004; Leontitsis & Pagge, 2007). In the literature, there is no agreement 


regarding the value of alpha in analyzing the reliability of an instrument. Along these lines, 
the lower limit of .7 is accepted and other studies accept .6 for exploratory research (Hair 
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Junior, Black, Babin & Anderson, 2009; Santos, 1999). Values lower than 0.6 indicate that 
the instrument has a low internal consistency, which can lead to incorrect conclusions about 
what it is supposed to measure (Hair Junior, Black, Babin & Anderson; Landis & Koch, 1977; 
Malhotra, 2011; Vieira, 2009). The authors Freitas and Rodrigues (2005) suggest a classification 
for alpha values, presented in Table 1. 


Table 1. Classification of the Cronbach's Alpha values and reliability. 








Reliability Very low Low Moderate High Very High 
iia BANE ge at 30<a< 60 60<a<.75 75<a< 90 9<a 





In the present research, the coefficient was analyzed per individual sample, including the 
overall Cronbach’s Alpha calculation for the entire instrument (considering all 28 statements) 
and by dimension (SMK, IRS, IOC and KSU). Moreover, calculations related to the exploratory 
analysis (mean, median and standard deviation) were carried out. 

The analysis of the instrument was carried out separately for each pre-service tacher. 
Primarily, it will be presented the treatment of the data related to the percentage of respondents 
who answered each question that constituted the categories. Then, it will be presented the 
exploratory analysis for the five pre-service teachers and finally, the presentation and discussion 
related to the Cronbach’s Alpha values and the validation of the instrument. The analysis is 
founded on the exploratory procedure and not the inferential procedure, meaning the set of data 
from the sample will not be used to generalize to a population (Bland & Altman, 1997). The 
analysis of the open question of the instrument will not be presented here. 


Validity 


The validity of an instrument analyzes whether it is evaluating what it is meant to 
measure. There are no formal mathematical methods to analyze the validity of a instrument, 
despite some studies appealing to a statistical analysis of factorial design, which describes 
the correlation between variables. The factorial analysis is considered controversial due to its 
subjectivity (Bittencourt, Creutzberg, Rodrigues, Casartelli & Freitas, 2011). As a result of this, 
researchers appeal to methods of reliability (Hora, Monteiro & Arica, 2010). However, there is 
no reciprocity between the instrument being reliable and valid, since not every construct that 
is reliable is valid. On the other hand, every valid instrument can be reliable (Vieira, 2009). 
Various authors propose different methods of validation (Bittencourt, Creutzberg, Rodrigues, 
Casartelli & Freitas, 2011; Vianna, 1976). Moron (1998) suggests the apparent validity of the 
content, criterion and construct. In this research, we only consider the validity of the content, 
which consists of an analysis of the content in the instrument by specialists from the area 
(Haynes, Richard & Kubany, 1995). 


Research Results 
Exploratory Analysis 


The pre-service teacher Anténio was in his freshman year in the PIBID teacher program 
and in the sixth semester of the teaching degree in chemistry. Anténio taught class in a 
secondary senior year class in EJA with 13 attending students. Table 2 gathers the percentages 
of the participants who completed the statements that constitute the categories Subject Matter 
Knowledge (SMK), Instructional Representation and Strategies (IRS), Instructional Objective 
and Context (IOC) and Knowledge of Students’ Understanding (KSU). 
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Table 2. Percentage of respondents who answered the instrument in each 
category — Antonio. 





Answers given (%) 





Answers given (%) 















































Statement 0 | 2 3 4 5 Statement 0 1 2 3 4 5 

SMK 1 0 15 38 46 1001 0 0 0 15 46 38 
SMK 2 0 0 0 46 31 23 1002 0 0 0 8 23 69 
SMK 3 0 0 0 38 = 23. 38S «1003 0 0 15 15 38 31 
SMK 4 8 0 0 0 62 31 1oc4 0 0 8 38 46 8 

SMK 5 0 0 0 23 54 «8923s (1005 0 0 15 23 23 38 
SMK 6 8 0 8 23 «31 «931 «1006 8 0 8 15 46 23 
SMK 7 0 0 8 0 54 38 =10C7 0 8 0 8 31 54 
IRS 1 0 0 0 15 38 46 KSU1 0 0 15 31 23 31 
IRS 2 0 8 8 8 54 23 KSU2 0 8 15 38 38 0 

IRS 3 0 0 8 31 31. 31. xKSU3 8 8 0 3146 8 

IRS 4 0 0 8 15 8 69  KSU4 8 8 0 38-23 23 
IRS 5 0 0 8 23. 31 += 38 =~ KSU5 0 0 8 8 54 31 
IRS 6 0 0 0 38 =662_—tisCOO KSU6 8 0 8 23-38 23 
IRS 7 8 0 31 15 31 15 KSU7 0 0 15 15 31 38 





0 (I don’t know), | (Never), 2 (Seldom), 3 (Sometimes), 4 (Often) and 5 (Always). 


In general, the mean, median and standard deviation for the categories are reflected in 
the percentage of students who completed the questions that constituted the categories and their 
scales. These values are presented in Table 3. 


Table 3. Mean and standard deviation values for each category referring to the 
answers about Antonio’s classes. 











Categories Mean Median Standard deviation 
SMK 3.98 4 0.99 
IRS 3.85 4 1.07 
OC 3.98 4 1.04 
KSU 3.54 4 1.21 





Melissa also developed her activities in EJA during the semester. The student-teacher 
was in her freshman year of the teacher education course and of PIBID. The instrument was 
answered by 12 students, while one of the students did not want to answer the instrument for 


Melissa. 


Primarily, the data were organized (Table 4) according to the percentage of respondents 
who completed one of the twenty-eight subcategories, in addition to the maximum and minimum 


values given. 
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constitute each category — Melissa. 


















































Answers given (%) Answers given (%) 

a 0 1 2 3 4 5 Statement 0 1 #2 3 4 ~« 5 
SMK 1 83 60 0 8.3 33.3 50 lOC 1 0 0 83 25 333 33.3 
SMK 2 0 0 8 3317 42 IOC 2 0 0 0 17-33 50 
SMK 3 8 0 0 17-33 42 OC 3 0 0 17 17-33 33 
SMK 4 0 0 0 17 8 75 lOc 4 8 0 8 7 «25 42 
SMK 5 0 0 0 25 42 33 lOc 5 8 8 8 33 33 
SMK 6 0 0 83 25 33.3 33.3. 10C6 177 0 8 8 42 25 
SMK 7 0 0 0 25 = 33 42 lOC 7 0 0 8 17 42 33 
IRS 1 0 0 8 25 25 42 KSU 1 8 17 8 17 25 25 
IRS 2 0 8 0 3317 42 KSU 2 0 8 8 25 25 33 
IRS 3 0 0 8 8 50 33 KSU 3 8 0 17 17-33 25 
IRS 4 0 8 8 17 25 42 KSU 4 0 8 8 17-33 33 
IRS 5 8 0 8 17-33 33 KSU 5 0 8 0 25 33 33 
IRS 6 0 17 8 17-33 25 KSU6 8 177 0 8 25 42 





IRS 7 8 8 8 25 25 25 KSU 7 0 0 8 25 25 42 
0 (I don’t know), 1 (Never), 2 (Seldom), 3 (Sometimes), 4 (Often) and 5 (Always) 





The grouping of the values for mean, standard deviation and median are presented in 
Table 5. 


Table 5. Mean values and standard deviation for each category referring to the 
answers about Melissa’s classes. 

















Category Mean Median Standard deviation 
SMK 4.09 4 1.07 
IRS 3.72 4 1.30 
lOc 3.80 4 1.30 
KSU 3.60 4 1.41 





The third pre-service teacher, Mateus, was in his sophomore year in the program PIBID 
and in the teacher education course. His classes in the partner school were given in two 
sophomore year classes in regular secondary school, which were usually held, due to the low 
attendance of the students in the teaching institution. Table 6 gathers the percentages of the 
participants who valued the statements that constitute the categories analyzed. 
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Table 6. Percentage of respondents who valued the statements, that constitute [75 


each category — Mateus. 















































State- Answers given (%) State- Answers given (%) 

ment o 41 #2 #3 4 + +=§ ment 4 2 3s «4. 4 
SMK 1 0 0 0 20 0 80 1OC 1 0 0 66 66 266 60 
SMK 2 0 66 860 20 60 13.3. 10C2 0 0 66 33.3 33.3 26.6 
SMK 3 0 0 0 13.3 666 20 1OC 3 0 0 0 13.3 333 53.3 
SMK 4 0 0 66 133 66 73.3 1004 0 0 66 266 33.3 33.3 
SMK 5 0 0 66 66 40 466 1005 0 0 66 266 266 40 
SMK 6 66 0 66 33.3 20 33.3 10C6 0 0 13.3 66 266 53.3 
SMK 7 0 0 66 66 533 333 1007 0 0 66 66 13.3 73.3 
IRS 1 0 0 0 20 13.3 66.6 KSU1 20 66 13.3 266 20 13.3 
IRS 2 0 40 0 33.3. 66 20 KSU 2 66 0 66 53.3 13.3 20 
IRS 3 0 13.3 0 13.3 0 73.3. KSU3 0 0 20 26.6 26.6 26.6 
IRS 4 0 66 0 66 66 80 KSU 4 66 0 13.3 33.3 20 26.6 
IRS 5 0 66 66 266 266 33.3 KSU5 0 0 66 20 6.6 66.6 
IRS 6 0 66 66 333 133 40 KSU 6 0 0 0 13.3 40 46.6 
IRS 7 0 13.3 666 66 66 66 KSU7 0 0 6.6 13.3 66 73.3 





0 (I don’t know), | (Never), 2 (Seldom), 3 (Sometimes), 4 (Often) and 5 (Always) 


The results of the instrument answered by the students for Mateus, regarding the 


exploratory analysis are presented in Table 7. 


Table 7. Mean and standard deviation values for each category referring to the 
answers about Mateus’ classes. 














Categories Mean Median Standard deviation 
SMK 4.12 4 0.98 
IRS 3.65 4 1.45 
OC 4.18 4 0.94 
KSU 3.74 4 1.35 





The PIBID pre-service teacher Clara was in her freshman year of the chemistry teaching 
education course and also in PIBID. She worked together with Mateus and Elder. Table 
8 presents the percentages of participants who completed the questions that constituted the 


categories analyzed. 
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76| Table 8. Percentage of respondents who valued the statements, which 





Answers given (%) 





Answers given (%) 












































Statement 0 1 2 3 4 5 Statement 0 4 2 3 4 5 
SMK 1 0 6.6 0 20 26.6 466 10C1 0 0 66 13.3 33.3 46.6 
SMK 2 0 6.6 6.6 6.6 26.6 533 l0C2 0 0 66 6.6 26.6 60 
SMK 3 0 6.6 0 6.6 26.6 60 IOC 3 0 0 66 20 13.3 60 
SMK 4 0 6.6 6.6 6.6 13.3 666 l0C4 0 0 66 26.6 20 46.6 
SMK 5 0 6.6 0 13.3 266 633 10C5 0 0 20 13.3 13.3 53.3 
SMK 6 0 6.6 13.3 6.6 26.6 466 l0C6 0 0 66 0 33.3 60 
SMK 7 0 6.6 0 13.3 20 60 lOC 7 0 0 133 0 40 46.6 
IRS 1 0 6.6 13.3 6.6 40 33.3. KSU 1 0 0 66 33.3 20 40 
IRS 2 0 20 6.6 13.3. 26.6 333 KSU2 266 0 6.6 20 20 26.6 
IRS 3 0 0 6.6 13.3. 13.3 66.6 KSU3 0 0 133 0 53.3 33.3 
IRS 4 0 0 6.6 0 26.6 66.6 KSU4 0 0 66 0 26.6 66.6 
IRS 5 0 0 13.3 20 20 46.6 KSU5 0 0 66 6.6 26.6 60 
IRS 6 0 0 13.3 20 33.3 33.3 KSU6 0 0 66 6.6 33.3. 53.3 
IRS 7 0 133 133 66 26.6 40 KSU 7 0 0 66 0 26.6 66.6 





0 (I don’t know), 1 (Never), 2 (Seldom), 3 (Sometimes), 4 (Often) and 5 (Always) 


The results of the instrument answered by the students for Clara, regarding the exploratory 
analysis are presented in Table 9. 


Table 9. Mean and standard deviation values for each category referring to the 
answers about Clara’s classes. 

















Categories Mean Median Standard deviation 
SMK 4.17 5 1.18 

IRS 3.96 4 1.23 

loc 4.23 5 1 

KSU 4.09 4 1.23 


Similarly to Clara and Melissa, the PIBID pre-service teacher Elder was in his freshman 
year of chemistry teacher education course. He entered PIBID in the second semester of 
the teacher education course. In his initial trajectory in the context of the beginning teacher 
program, he taught classes on the redox content. Table 10 presents the percentage of students 
who answered the questions that constitute each category of teacher knowledge. 
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Table 10. Percentage of respondents who valued the statements that constitute 





















































each category — Elder. 

Answers given (%) Answers given (%) 
Statement stale: 

0 1 2 3 4 5 ment 0 1 2 3 4 5 
SMK 1 0 66 66 13.3 133 60 IOC 1 0 0 6.6 266 466 20 
SMK 2 0 0 66 20 33.3 40 lOC 2 0 0 66 13.3 266 53.3 
SMK 3 0 0 133 66 20 60 lOC 3 0 0 20 13.3. 13.3 53.3 
SMK 4 0 0 66 133 266 53.3 1004 0 66 66 20 333 533 
SMK 5 0 0 66 20 26.6 466 1005 0 0 20 66 333 40 
SMK 6 0 66 133 66 266 466 I0C6 66 0 66 266 20 40 
SMK 7 0 66 66 20 666 1007 66 66 O 66 466 333 
IRS 1 66 8600 13.3. 13.3 40 266 KSU1 0 0 26.6 20 13.3 40 
IRS 2 0 133.66 266 20 333 KSU2 66 66 66 266 266 266 
IRS 3 0 0 20 0 40 40 KSU3 0 66 0 13.3. 40 40 
IRS 4 0 0 66 20 26.6 466 KSU4 0 13.3 0 20 = 33.3 33.3 
IRS 5 0 66 66 66 333 466 KSU5 0 0 0 26.6 33.3 40 
IRS 6 66 60 133 66 333 40 KSU6 0 0 0 20 466 33.3 
IRS 7 0 66 20 20 13.3 40 KSU7 0 0 0 20. 20 60 





0 (I don’t know), 1 (Never), 2 (Seldom), 3 (Sometimes), 4 (Often) and 5 (Always) 


Table 11 presents the percentage of students who valued the statements that constitute 


each category of teacher knowledge. 


Table 11. Mean and standard deviation values for each category referring to the 
answers about Elder’s classes. 

















Categories Mean Median Standard deviation 
SMK 4.18 5 1.07 
IRS 3.82 4 1.28 
10C 3.90 4 1.19 
KSU 3.93 4 1.13 





The measures of exploratory statistical analysis did not indicate the reliability of the 
instrument. As such, it was calculated Cronbach’s Alpha to ensure the reliability of the answers 
given by the students regarding their perceptions of the PCK of the pre-service teachers. In 
Table 12, the values of the alpha coefficient were compiled for the entire instrument and by 


category. 
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Table 12. Cronbach's Alpha values for the entire instrument and by category. 





Pibid students Alpha values 























(pre-service — ategory— ttst—“‘iOSOSO™S™S*S*S*S*S*SS 
meaehiers) memument SMK IRS loc KSU 

Anténio 92 69 18 12 83 

Melissa 95 AT 82 vai 91 

Mateus 95 85 84 89 JT 

Clara 96 97 82 89 84 

Elder 94 84 86 83 80 

Validation 


In this research, the validation of the content was done by 3 graduate students who 
research chemistry teacher knowledge. 


Discussion 
Exploratory Analysis 


Data analyses of the Table 2, in the SMK 1 question “My teacher knows the content 
he/she is teaching’, for 46%, Antodnio always demonstrated proficiency in the redox subject 
that he was teaching and, for another 31%, the student-teacher frequently explained the 
impact of this subject on society (SMK 6). In the item IRS 7 “My teacher uses multimedia or 
technology to express the concepts”, Anténio rarely (31%) used technology as a strategy to 
develop concepts. However, the same percentage of students believed that Anténio frequently 
used these resources. Despite the students recognize that Antonio used different approaches, 
the use of strategies involving media and technology perhaps needs to be improved. Regarding 
the category IOC 2 “My teacher provides an appropriate interaction or good atmosphere” for 
69% of the studentes, Anténio always provided interaction with the students and, consequently, 
made the classroom space conducive to a social atmosphere. Upon contrasting this percentage 
with another similar question from the IRS 4 category “My teacher provides opportunities for 
me to express my views during class”, it was observed in the paragraph above that the same 
percentage is obtained from students who answered that this atmosphere a/ways existed in the 
student-teacher classes, indicating that the items involved in these questions are correlated. 

In the item KSU 6 “My teacher s assignments facilitate my understanding of the subject’, 
in the view of 38% of the class, the use of a determined methodology often facilitated the 
understanding of the subject, while for 23%, it always happened, and, for another 23%, it 
sometimes occurred in the teaching practice. Perhaps it is difficult for the students to understand 
the extent of the activities developed by Anténio before going into the classroom and, 
consequently, to understand that one of the results of these tasks is class preparation. 

The results presented in Table 3, the mean values found for the categories do not have 
significant differences, suggesting that the students marked primarily scale 4 (Frequently 
happened in the teaching practice). In relation to the standard deviation, the category with the 
lowest value was the SMK and that with the highest value is KSU. The standard deviation 
values higher than one indicate that the students did not consistently answer the questions that 
constituted the categories. On the other hand, a value lower than one means that the groups 
marked mainly two scales, revealing consistent student perceptions. 
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Based on the exploratory analysis, it was understood that, according to student 
perceptions, Antonio frequently (scale 4) demonstrated knowing the concepts that he taught, 
enabling the students to understand the objectives of his classes, the strategies that helped them 
in teaching and learning the specific subject, in addition to demonstrating an understanding of 
the difficulties, limitations and needs of his students in relation to the specific subject. 

Upon analyzing the data for Table 4, in the subcategory SMK 1 “My teacher knows the 
content of teaching”, according to 50% of the students from EJA, Melissa always demonstrated 
knowledge of the content, while 8% answered I don’t remember/I don’t know. In SMK 4 “My 
teacher selects the appropriate content for students” for 75% of the students, the student- 
teacher always adequately chose the concepts. On the other hand, 25% answered sometimes 
and for another 25% Melissa frequently used adequate examples to teach (IRS 1). In relation 
to using analogies in the student-teacher classes (IRS 2), for 42% of the students, they were 
always in Melissa’s classes, while for another 33%, it sometimes occurred in her practice, which 
generally means that the students did not always observe Melissa using analogies in her classes. 

In relation to the IOC 1 subcategory “My teacher makes me clearly understand objectives 
of this course” for 8% rarely, 25% sometimes, 33% frequently, meaning that for 66% of the 
students, Melissa did not always make her educational objectives clear. When questioned on IOC 
6 “My teacher copes with our classroom context appropriately” in the perspective of students 
17% answered J dont remember/I dont know, 8% seldom, 8% sometimes and 42% answered 
frequently. These values lead to the inference that the students did not have a homogeneous 
perception of how Melissa coped with the class. 

In KSU 1 “My teacher realizes students’ prior knowledge before class”, for 25%, Melissa 
always checked their previous knowledge, while the perception for 25% was frequently. By 
contrast, 8% answered J dont remember/I dont know, 17% never happened in practice, 8% 
rarely happened and for 17%, sometimes. 

In the Table 5, the standard deviation values for the four categories are higher than 
1, which means that the students did not have a homogeneous perception of the statements 
constituting SMK, IRS, IOC and KSU. The values of the means are very similar, which was 
reflected in a median of scale 4 (frequently). These data suggest that in the students’ perception, 
Melissa frequently accessed during her classes the knowledge related to the redox content and 
to teaching strategies, among others. The standard deviation values for the four categories 
are higher than 1, suggesting that the students had different perceptions for the statements 
constituting SMK, IRS, IOC and KSU. The mean values are very similar, which reflected in 
a median of 4 (If it happened frequently in the teaching practice), which means that in the 
perception of the group, Melissa frequently accessed during her classes the knowledge related 
to the redox content and to teaching strategies, among others. 

According to the data presented in Table 6, in question SMK 1 “My teacher knows the 
content of teaching”, for 80% of the class, Mateus always seemed to dominate the content on 
electrochemistry that he taught. Perhaps knowing that the PIBID student-teacher dominated the 
subject, when questioned SMK 3 “My teacher knows how theories or principles of the subject 
have been developed” for 66%, the pre-service teacher often demonstrated knowing them. In 
relation to the use of analogies (IRS 2), 40% of the students marked scale | (If it never happened 
in the teaching practice) which means that they did not perceive the use of this strategy. On the 
other hand, for 66.66% of the respondents, the PIBID student-teacher gave examples (IRS 1) 
that made the learning easier. 

In relation to IOC 2 “My teacher provides an appropriate interaction or good atmosphere” 
the class did not have a hegemonic perception, since, for 6.66% it rarely happened, for 33.33% 
sometimes, for 33.33% often and, for only 26.66% Mateus was always able to implement a 
good atmosphere for interpersonal development. However, for 70% of the students, the PIBID 
student-teacher always promoted dynamic and active teaching (IOC 7). 

The students’ perception of the knowledge of the students’ understanding reveals that only 
13.33% indicated that Mateus always demonstrated his prior knowledge (SMK 1) Regarding 
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the evaluation, in question KSU 7 “My teacher 5 tests help me realize the learning situation’, for 
73.33%, they always contributed to the students’ understanding the electrochemistry concepts 
that formed the foundation of Mateus’ classes. In relation to the teaching approaches in the 
classes of the PIBID pre-service teacher’s, when they analyzed question KSU 5 “My teacher 
uses different approaches (questions, discussion, etc.) to find out whether I understand’, 66.6% 
of the students were able to identify the various teaching approaches and strategies that involved 
teaching the specific content. 

In the Table 7 IRS and KSU categories, the standard deviation is higher than 1, which 
reveals that the group of students had a more heterogeneous perception about the knowledge 
involving teaching strategies and the students’ knowledge in the classes taught by Mateus. On 
the other hand, it was observed a homogeneity in the answers for the statements from the SMK 
and IOC categories, suggesting that the class had a similar perception of the statements that 
constituted Mateus’ knowledge in relation to the content, the class and his teaching proposals. 

In her trajectory in PIBID, Clara helped the students with their doubts in relation to the 
chemistry content presented by Elder and Mateus. According to the results of Table 8, when 
the students were questioned regarding SMK 2 “My teacher explains clearly the content of 
the subject” and SMK 5 “My teacher knows the answers to the questions that we ask about 
the subject’, the same percentage of students, 53.3%, indicated that Clara was always able 
to explain their doubts and knew the electrochemistry content. Perhaps this perception of the 
students was a reflection of the PIBID student-teacher providing space to express their points of 
view (IRS 4) according to 66.6% of the students. On the other hand, for 26.6% of the students, 
Clara did not previously know about students’ difficulties regarding electrochemistry (KSU 2). 

In relation to the values of the exploratory analysis (Table 9), the standard deviation was 
higher than 1 for the SMK, IRS and IOC Categories, revealing an inconsistent perception of 
the students about this knowledge in the PCK of the PIBID pre-service teacher’s. On the other 
hand, in the IOC category, which represented knowledge related to the strategies for teaching 
the content, the students demonstrated more similar perceptions. 

The categories of teacher knowledge presented a standard deviation higher than or equal 
to one, which leads us to infer that the students had a more heterogeneous view on the questions 
that constituted the categories of Clara’s knowledge. 

Acording to the results of Table 10 in category SMK 1 “My teacher knows the content he/ 
she is teaching” for 60% of the students, Elder had knowledge about redox reactions. When the 
students analyzed the examples chosen by the PIBID student-teacher to explain the concepts 
URS 1), for 40% of the students, the examples often made it easier to understand the concepts 
taught. For 40% of the class, the PIBID student-teacher a/ways had an adequate relation with 
the class (IOC 5), while for 20% of the class, it happened rarely. 

In statement KSU 2 “My teacher knows students’ learning difficulties of subject’, for 
26% of the students, the PIBID student-teacher was sometimes, often and always aware of 
his limitations with the redox subject. These data suggest that Elder needs to improve his 
understanding of the students’ learning difficulties regarding the concepts he teaches. 

The standard deviation values presented in Table 11 for the four categories raised a 
diverging view of the students on Elder’s teaching knowledge in relation to redox reactions, 
students’ difficulties, teaching strategies, the context, among others. 


Reliability - Cronbach’ Alpha 


The results presented in Table 12 for Alpha from pre-service teachers were: .92 (Antdénio), 
.95 (Melissa), .95 (Mateus), .96 (Clara) and .94 (Elder) reveal that the set of answers in the 
instrument used presented reliable answers and, consequently, very good internal consistency. 
The literature points out that the reliability of an instrument can present very good consistency 
in relation to the context. From this, one can infer that the degree of correlation between the 
statements that constitute each category had a good correlation as a result of the alpha values, 
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since the number of statements in the instrument could result in an increase in the consistency 
of the scale used (Krus & Helmstadter, 1993; Streiner, 2003). For Vieira (2009), when the alpha 
is calculated item by item, as was done in this research, there is a decline or improvement in the 
overall consistency, when a determined item is important or not in the instrument. 

The calculation of the alpha coefficient for each of the dimensions (SMK, IRS, IOC and 
KSU) are also a consequence of this. The values for the four dimensions of teacher knowledge 
are higher than .6, which suggests that the instrument as a whole presents a good correlation 
between the categories. Moreover, the categories can be implemented separately if the researcher 
wished to do so. However, it was considered more pertinent to use it as a whole, as done in this 
research, since there are statements in each category that were related to the other categories, 
when considering its individual characteristics. 

Overall, the reliability calculated for the instrument and analyzed for each participant 
revealed that the instrument is reliable and produces stable and consistent measurements, 
meaning that the instrument is reliable when used in the context of early teacher training. 


Validation 


Since the graduate students’ analyses were effective, some commented that the statements 
in the categories were clear. On the other hand, others believe that certain questions bring up 
concepts that perhaps the students cannot understand, such as, for example, what the prior 
knowledge of the students is, analogies and methodologies. It is worth noting that, upon 
applying this instrument, these terms were explained to the students. Another suggestion would 
be to apply the instrument and change the order of the statements of each category. It is worth 
pointing out that this was not done in the present research. 


Conclusions 


The aim of this research was to evaluate the reliability of the instrument which measures 
teacher knowledge, in the context of PIBID and, consequently, whether the instrument could be 
used to measure the teacher’s knowledge in pre-service teacher, based on student perspectives. 

The values found for the Cronbach’s Alpha allow us to conclude that the instrument 
applied is reliable regarding students’ perceptions of the teaching knowledge of the PIBID pre- 
service teacher's. Therefore, the instrument can be used in contexts of early teacher training 
and reveal to these subjects the specific components of professional knowledge of the pre- 
service teachers that the students did not perceive. This information can be valuable for pre- 
service teachers and can influence the planning and implementation process of classes, as 
well as student learning. It was believed the instrument can be used in research related to pre- 
service teachers and continued teacher training, such that these professionals can reflect on their 
practice, based on their students’ perceptions, because in the previous researches in which they 
were applied the instrument involved analysis of teacher knowledge in the university professors 
and experienced teachers. 

In general, the purpose of this research was reached, because it was possible to assess 
the reliability of an instrument that measures teacher knowledge in the context of pre-service 
teachers. The future goal of the research is to adopt the instrument in activities developed by the 
pre-service teachers so that their planning of the didactic sequences contemplate the four SMK, 
IRS and IOC categories of the instrument. In addition, our intention is to apply the instrument 
to evaluate the teacher knowledge of pre-service chemistry teachers through the establishment 
of partnerships with researchers of the teaching area linked to other universities in the city of 
Sao Paulo. We believe that the adoption of this instrument in several stages of the education of 
undergraduates may promote reflective processes that contribute to the continuous professional 
development of these future professionals. 
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Appendix 


This instrument aims to recognize and record the perceptions of high school students 
about the PCK of PIBID pre-service teacher's. This instrument is an adaptation of the proposal 
of Jang, Guan and Hsieh (2009). 

This set of statements aims to recognize and record the perceptions of high school 
students about the PCK of PIBID pre-service teacher's. 

Analyze the statements that follow and use the scale below to express how are your 
views to it, considering the Pibid pre-service teacher's , during the 
implementation of their classes this year. Be sure to give an scale for all statements. Mark the 
scale that corresponds to your perception. 








| do not remember / | do not know 

Never occurred never in teaching practice 
Seldom occurred in teaching practice 
Sometimes occurred in teaching practice 
Often occurred in teaching practice 
Always occurred in the teaching practice 
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Scale 
Category Statements 
0 1 2 3 4 °5 
Subject Matter 1. My teacher knows the content is teaching. 
-— 2. My teacher explains clearly the content of the subject. 
3. My teacher knows how theories or principles of the subject 
have been developed. 
4. My teacher selects the appropriate content for students. 
5. My teacher knows the answers to questions that we ask about 
the subject. 
6. My teacher explains the impact of subject matter on society. 
7. My teacher knows the whole structure and direction of this 
SMK. 
Instructional 1. My teacher uses appropriate examples to explain concepts 
Representation & related to subject matter. 
Strategies (IRS) “2. My teacher uses familiar analogies to explain concepts of 


subject matter. 

3. My teacher's teaching methods keep me interested in this 
subject. 

4. My teacher provides opportunities for me to express my views 
during class. 

5. My teacher uses demonstrations to help explaining the main 
concept. 














6. My teacher uses a variety of teaching approaches to transform 
subject matter into comprehensible knowledge. 





7. My teacher uses multimedia or technology to express the 








concept of subject. 
Instructional Ob- 1. My teacher makes me clearly understand objectives of this 
jective & Context course. 
(10C) 2. My teacher provides an appropriate interaction or good 
atmosphere. 





3. My teacher pays attention to students’ reaction during class 
and adjusts his/her teaching attitude. 





4. My teacher creates a classroom circumstance to promote my 
interest for learning. 


5. My teacher prepares some additional teaching materials. 





6. My teacher copes with our classroom context appropriately. 





7. My teacher's belief or value in teaching is active and aggres- 
sive. 





Knowledge of 
Students’ Under- 
standing (KSU) 


1. My teacher realizes students’ prior knowledge before class.. 


2. My teacher knows students’ learning difficulties of subject 
before class. 

3, My teacher's questions evaluate my understanding of a topic. 
4. My teacher’s assessment methods evaluate my understand- 


ing of the subject. 











5. My teacher uses different approaches (questions, discussion, 
etc.) to find out whether | understand. 





6. My teacher's assignments facilitate my understanding of the 
subject. 
7. My teacher's tests help me realize the learning situation. 
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Comments: 


During the pre-service teacher class: 


(name of the pre-service teacher) ; 





do you have any learning difficulty? If you want to comment on the student teacher, please use 








this space. 
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